To solve the problem of huge computation cost and low matching accuracy in the course of iterative calculation while using RANdom SAmple Consensus (RANSAC) algorithm for image splicing, a mismatching elimination algorithm was proposed based on motion smoothing constraint terms. Firstly, feature points were extracted with ORB (Oriented FAST and Rotated BRIEF) algorithm, and initial matching of feature points was implemented based on Hamming distance. Secondly, the statistical neighboring support estimators based on motion smoothing constraint terms were used to achieve rough mismatching elimination, and then spatial geometric constraints were applied to refine mismatching elimination. Finally, grouping sorting was used to solve the model parameters, and weighted averaging was used to realize image fusion. The experimental results show that the mismatching elimination rate is improved by 75.6% compared to the algorithm for reducing the total number of sampling points and 24% compared to adaptive threshold algorithm. This method can effectively eliminate mismatching and realize accurate image mosaic.